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ABSTRACT 

The Internal Fluid Mechanics Division of the NASA Lewis Research Center 
is combining the key elements of computational fluid dynamics, aerothermody- 
namic experiments, and advanced computational technology to bring internal 
computational fluid mechanics (ICFM) to a state of practical application for 
aerospace propulsion system design. This paper presents an overview of efforts 
underway at NASA Lewis to advance and apply computational technology to ICFM. 
These efforts include the use of modern, software engineering principles for 
code development, the development of an Al-based user-interface for large 
codes, the establishment of a high-performance, data communications network to 
link ICFM researchers and facilities, and the application of parallel process- 
ing to speed up computational ly-intensive and/or time-critical ICFM problems. 

A multi-stage compressor flow physics program is cited as an example of efforts 
to use advanced computational technology to enhance a current NASA Lewis ICFM 
research program. 

INTRODUCTION 

The development of advanced aerospace propulsion systems, that must be 
lighter, more durable, more efficient, and cost-competitive, demands increased 
understanding of all of the physical phenomena (e.g., coupled fluid and struc- 
tural mechanics with three-dimensional viscous flows, turbulence, shock- 
boundary layer interactions, combustion, heat transfer, multi-stage rotating 
machinery, flexible structures, high-temperature materials) that determine per- 


formance and life. 


The National Aeronautics and Space Administration is committed to carrying 
out research and technology programs to acquire basic understanding of the fun- 
damental physics in propulsion systems and to build this understanding into 
numerical simulations that can be used to identify promising regions of propul- 
sion system "design-space" for commercial exploitation. 

As revolutionary changes take place in the designs of propulsion system 
components, the need for high-fidelity computer simulations becomes more acute 
because past experience and data cannot be extrapolated with confidence. Also, 
it may be prohibitively expensive to fabricate, assemble, and test each new 
concept and design under realistic conditions. In some cases (e.g.. National 
Aerospace Plane), experimental facilities might not even exist to test new com- 
ponent and system designs. 

The need exists, therefore, for computer simulations that can be used as 
"numerical test cells." In the case of computational fluid dynamics (CFD), 
however, very few simulation codes accurately portray the three-dimensional 
flow physics that takes place within real propulsion systems - either because 
of a lack of understanding of the fundamental physics or a lack of computing 
power (speed and/or memory) to solve the model equations in a timely fashion. 

As a result, propulsion system development remains a heuristic process, involv- 
ing the use of highly empirical models and extensive hardware testing. 

As progress is made in gaining a fuller understanding of the complex fluid 
dynamics in propulsion systems, we need to match that understanding with suffi- 
cient computational capability to allow the practical use of internal computa- 
tional fluid mechanics (ICFM) codes as engineering and design tools. This 
implies numerical solutions and presentation of results to the analyst in min- 
utes or hours, rather than days or weeks. Also, the scope of the simulation 
task must, in many instances, go beyond the calculation of only the fluid 
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behavior in a single component to include the interactions of fluid and 
structural mechanics (i.e., aeroelasticity) and the interactions of various 
components in the overall propulsion system. This represents a tremendous com- 
putational challenge that will only be met through continued advances in (1) 
computer science and technology, and (2) the ability of the user community to 
exploit those advances. 

This paper presents an overview of efforts underway at NASA Lewis to 
develop, adapt, and apply computational technology to propulsion CFD. These 
efforts include: 

(1) The development of PROTEUS, a three-dimensional Navier-Stokes test- 
bed flow solver. PROTEUS represents a novel approach to CFD code development 
that features a multi-disciplinary development team, employing modern software 
engineering techniques. The emphasis is on creating a user-friendly code that 
can be tailored to a variety of propulsion applications. 

(2) The development of PROTAIS, an intelligent user-interface to the 
PROTEUS code. PROTAIS is a knowledge-based system that is being developed in 
collaboration with the PROTEUS code developers. The goal is to provide 
computer-based guidance and assistance to both expert and novice users of the 
PROTEUS code. 

(3) The establishment of a high-performance, data communications network 
(ERBNET) that provides researchers with access to a wide spectrum of computing 
facilities, including local (departmental), centralized (Lewis-wide), and 
remote (e.g.. Numerical Aerodynamic Simulator at NASA Ames) computer systems. 

(4) In-house and sponsored research and development activities aimed at 
exploiting PARALLEL PROCESSING architectures for propulsion CFD. These activi- 
ties range from multi-tasking of codes on the CRAY X/MP supercomputer to the 
buildup and testing of the HYPERCLUSTER, an experimental parallel processing 
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workstation. Hardware and software strategies are being sought that can 
deliver fast, interactive, and affordable computing to the CFD researcher. 

(5) Focused efforts to integrate and apply the aforementioned technolo- 
gies and capabilities to enhance current and future CFD research programs. A 
proposed multi-stage compressor flow physics program is cited as an example. 

ICFM RESEARCH THRUSTS 

The Internal Fluid Mechanics Division ( IFMD) at NASA Lewis is combining 
the key elements of computational fluid dynamics, aerothermodynami c experi- 
ments, and advanced computational technology to bring internal computational 
fluid mechanics (ICFM) to a state of practical application for aerospace pro- 
pulsion system design [1]. 

By focusing on specific portions of the propulsion system, IFMD research- 
ers are working to isolate and study the dominant phenomena in order to gain 
the understanding needed to develop an accurate, predictive capability. As 
shown in Fig. 1, the ICFM program is organized with Inlets, Turbomachinery, 
and Chemical Reacting Flows as the main research thrusts. 

The development of accurate, predictive propulsion CFD codes requires a 
close coupling of the numerical code development work and fundamental physical 
experiments. As shown in Fig. 2, each discipline is dependent on the other 
for information and guidance. The insights gained through fundamental physi- 
cal experiments form the basis for new mathematical models and codes. The 
resultant codes must be tested by comparison of analytical and experimental 
data, often leading to new test requirements. 

While the speed and memory capacity of computers have increased dramati- 
cally over the past 40 years, computer technology still remains a limiting 
factor in the quest to use high-fidelity computer simulations as tools in the 


4 



propulsion system design process. Code developers must determine the appropri- 
ate level of model detail that will, when combined with available computational 
power, satisfy the needs of the intended appl ication(s) . 

As an example, consider the range of computational tools envisioned for 
turbomachinery design and analysis. Figure 3 shows the relative levels of com- 
plexity of these tools, ranging from the unaveraged Navier-Stokes equations 
down to quasi-one-dimensional equations. The aggressive design of multi-stage 
turbomachines for the next century's propulsion systems will utilize complex 
geometries with endwall contouring and swept blading. Resulting flow fields 
will be highly three-dimensional with unsteady rotor-stator interactions and 
large secondary flows. It's clear that three-dimensional, unsteady, viscous 
codes will be needed to describe the physics. The same conclusions can be 
drawn for the other propulsion system components, as well. However, the use 
of high-fidelity ICFM codes requires a great deal of computer power. Figure 4 
indicates that it will take a 10 Gword/100 Mflop machine to obtain a time- 
accurate, Reynolds-averaged flow field solution, for a hypersonic inlet, in 
1 CPU hr. In a design process, where repetitive running of codes is required 
to perform sensitivity studies and parameter optimizations, it will require 
computers that have 100 gigaword in-core memories and Tflop speeds. Until such 
machines are readily available for propulsion system analysis, system develop- 
ers will be forced to continue using empirical models. Therefore, an important 
byproduct of the efforts to develop more accurate, two- and three-dimensional 
codes is the building up of analytical and experimental databases that can be 
used to extend the range of the empirical models. 

APPLICATION OF NEW COMPUTATIONAL TECHNOLOGY 
While supercomputer speed is critical, there are many other computer tech- 
nologies that can contribute to making ICFM practical for propulsion system 
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design. As shown in Fig. 5, the code development process can also benefit from 
advances being made in software engineering, expert systems, networks, and par- 
allel processing. If used properly, these technologies can be combined to form 
an integrated, cooperative environment for conducting ICFM analyses and experi- 
ments. The resulting databases, codes, and insights can, in turn, form the 
basis for a more confident, more aggressive design methodology. 

The following paragraphs describe efforts underway to develop and demon- 
strate advanced computational technologies for propulsion CFD. 

Software Engineering (PROTEUS) 

Numerous efforts in CFD consist of modifying existing programs to handle 
geometries and flow regimes of current interest to the researcher. Unfortu- 
nately, nearly all nonproprietary programs were started as research projects 
with emphasis on demonstrating a numerical algorithm rather than ease of use 
or modification. The end result is usually a haphazard collection of poorly 
documented, extensively patched code that is difficult to understand and which 
may not even work under certain conditions. 

Current efforts are directed at minimizing the above problems by stan- 
dardizing the structure of newer codes. One such code is PROTEUS, a general- 
purpose, three-dimensional Navi er-Stokes flow solver. By rigorously adhering 
to the software engineering principles that are shown in Fig. 6, the PROTEUS 
developers are producing a code that has a consistent, clear structure and is 
easy to understand and modify. 

Figure 7 illustrates the development cycle for the PROTEUS code. One of 
the more unusual aspects of the PROTEUS development has been the team approach 
to implementation. By specifying each program module in terms of its function 
and interfaces, each member of the multi-disciplinary development team has been 
able to independently code modules while treating the other modules as "black 
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boxes." This approach has permitted programming assignments to be made on the 
basis of individual skills and experience and has led to a more timely imple- 
mentation and testing of modules. 

From the beginning of the PROTEUS project, the developers recognized the 
critical importance of documentation. Internal program documentation is pro- 
vided by the liberal use of comment statements and header sections for each 
module. External documentation is provided by the creation of dictionaries 
that define input variables, common block variables, default variables, and 
module functions. 

Artificial Intelligence (PROTAIS) 

The application of software engineering principles to the PROTEUS develop- 
ment represented a big step toward making large CFD codes user-friendly. That 
effort has also provided a stimulus to researchers at NASA Lewis to investi- 
gate the use of artificial intelligence (AI) technology as an additional aid 
in developing and using such codes. The NASA Lewis effort attempts to capture 
the expertise gained by code developers in an "intelligent interface" that will 
allow others to make more effective use of the codes. 

The interface is initially being developed to support the PROTEUS code 
and, as such, has been named PROTAIS to denote the introduction of AI tech- 
niques to PROTEUS [2]. As shown in Fig. 8, the PROTAIS system is intended to 
provide help and advise for both expert and novice users of PROTEUS. It fea- 
tures a constantly growing knowledge base that reflects the experience and 
knowledge gained as the PROTEUS code is developed and run under a variety of 
condi tions . 

The PROTAIS development is being done on a Symbolics workstation (Fig. 9). 
Work, to date, has concentrated on developing expert user facilities that auto- 
mate many of the tedious and time-consuming aspects of setting up and running 


7 


PROTEUS. The user can query the interface to obtain information on running the 
code including definitions and status of existing code configurations (i.e., 
master sets, problem sets, ranges of input variables and their default values, 
cases run and results obtained, and connected computer systems and their proto- 
cols). At this writing, testing of the software design is being carried out 
in a stand-alone mode (simulating the interface to PROTEUS) on the Symbolics 
workstation. Later this year, local-area network facilities (see description 
of ERBNET) will be used to connect the interface, installed on general-purpose 
engineering workstations, to the PROTEUS code, running on a mainframe computer. 
The interface will then undergo extensive testing and evaluation by the PROTEUS 
developers. 

Future refinements to the PROTAIS interface will be aimed at novice users 
of the code, shielding those users from the details of the code and the envi- 
ronment under which it is running (i.e., computers, operating systems, net- 
works, etc.). Rules (or other knowledge representations) used by CFD experts 
will be added to the interface to guide the novice user in effectively using 
the code. 

Networks (ERBNET) 

In 1985, work began on the design of a local-area network (ERBNET) that 
would allow ICFM researchers to access and communicate between various termi- 
nals, computers, and workstations located in the Engine Research Building 
(ERB) complex. The network would also provide connectivity and high-speed 
data communications between the local computers, centralized mainframe comput- 
ers, and remote computers such as the NAS. 

In designing the ERBNET system, special attention was given to system 
adaptability, flexibility, and growth potential. The decision was made to use 
proven and available technology (Ethernet communications and TCP/IP networking 
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protocols) to reduce cost, allow rapid implementation, and permit modular — 
upgrades . 

Figure 10 shows a block diagram representation of the ERBNET configu- 
ration. A baseband Ethernet cable was installed to provide high-speed 
(10 Mbits/sec) connections between local computers and workstations. A utility 
band on the Lewis-wide broadband cable (LINK) was used to interconnect ERBNET 
with computers in the central Research Analysis Center (RAC) and to provide 
user access to the NAS facilities through the NASA Program Support Communica- 
tions Network (PSCN). Figure 11 shows one of the Silicon Graphics IRIS (Inte- 
grated Raster Imaging System) devices that is being used as a NAS workstation. 
The IRIS is capable of transforming and displaying three-dimensional data at 
rates exceeding 50 000 points per second. 

ERBNET continues to be expanded to provide additional capabilities to 
NASA Lewis researchers. ERBNET is being tied into several off-site networks to 
allow sharing of resources and information with university and industrial 
researchers. Additional equipment is being added to the network, including 
additional high-performance graphics workstations. Work is underway to develop 
graphics and scientific database software that will facilitate the interchange 
and display of data across the breadth of ERBNET devices. 

Parallel Processing 

Advancements in solid-state electronics and circuit design technology pro- 
duced nearly linear increases in computer speeds from 1950 to 1970. Since that 
time, computer manufacturers have had to look toward new computer architectures 
(e.g., vector and parallel processors) to gain additional computing speed. A 
modern supercomputer will have 2 to 8, bus-connected, vector processors with 
shared memory. When run in a parallel processing mode, these supercomputers 
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achieve benchmark speeds of 100 to 300 Mflops. As the pursuit of Tflop comput 
ers continues, the introduction of new, hi ghly-paral 1 el architectures poses 
significant challenges (and opportunities) for ICFM algorithm and code develop 
ers who must devise methods for harnessing the power of these machines. 

The calculation of three-dimensional, unsteady flow fields within multi- 
stage compressors and turbines is a prime candidate for parallel processing 
because of (1) the long run times that will be required to perform the multi- 
stage calculations and (2) the many levels of parallelism that do exist in cur 
rent multi-stage algorithms. Mulac, et al . , have demonstrated a technique for 
using the parallel processing capabilities of today's supercomputers for turbo 
machinery CFD [3]. An "average-passage" model that can compute flow fields in 
up to 20 blade rows (10 stages) was developed. As shown in Fig. 12, the 
average-passage model has been tested on a 4-processor CRAY X-MP with each 
blade row of a 2-stage turbine assigned to a separate CPU. The continuity, 
momentum, and energy equations for each blade row are computed in parallel. 

The effects of body forces (the other blade rows) on each blade row solution 
are handled with stress-like terms. While the closure modeling still remains 
to be determined, the solutions have demonstrated an ability to correctly 
account for blade row interactions. Figure 12 shows the computed steady, 
total temperature field within the four blade rows of the SSME fuel turbine. 

While supercomputers represent the only possibility for performing many 
"leading edge" CFD calculations, the high cost of supercomputers and the need 
to share supercomputing resources among large numbers of users make supercom- 
puters unavailable and/or impractical for many applications. Fortunately, a 
growing number of computer manufacturers have begun to combine parallel archi- 
tectures and state-of-the-art microprocessors to form a new class of machine - 
the mini supercomputer. Mi ni supercomputers , employing 10's (and even 100's) of 
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processors, can be purchased for less than $ 1 M , delivering 10 to 20 Mflop 
performance. It is now possible for many more groups of researchers and 
departments to become involved in the application of parallel processing to 
propulsion CFD and to begin to deal with the issues of scaling (i.e., adding 
processors to gain computing speed for a given problem). A number of studies 
are currently underway using both shared-memory and distributed-memory mini su- 
percomputers to solve ICFM algorithms [4-5], 

A three-pronged approach is being used at NASA Lewis to apply parallel 
processing to ICFM. First, supercomputers continue to be used to study tech- 
niques for parallel processing of large codes. Second, a mi ni supercomputer 
is being purchased and will be available, via the ERBNET, in late CY88. The 
commercial mi ni supercomputer will provide an excellent training ground for 
researchers working to adapt existing algorithms and/or develop new algorithms 
for parallel processing. Finally, a reconfigurable, parallel processing work- 
station, called the "Hypercluster," is being constructed and is due to be oper- 
ational in early CY89 [6]. The Hypercluster combines distributed memory 
(hypercube interconnections) and shared-memory (bus connections at each hyper- 
cube node) to provide a unified, cost-effective means of implementing and 
testing a variety of simulator configurations, and of tailoring a simulator 
architecture to match a particular algorithm. The initial Hypercluster config- 
uration will be two-dimensional with four nodes and four processors at each 
node (Fig. 13). 

As illustrated in Fig. 13, the Hypercluster will provide an opportunity 
to extend parallel processing of the average-passage turbomachinery model 
beyond 1-processor/blade row. Several approaches are being considered. In 
one, each cluster of four processors would be devoted to a particular blade row 
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calculation. This raises the question: "How best do you divide up the blade 

row calculations among the four processors?" 

In the case of an explicit algorithm (i.e., where each grid point solution 
only depends on past values at neighboring grid points), partitioning of the 
blade row solution grid may not be necessary. Milner has proposed a "time par- 
titioning" scheme in which each processor is calculating the entire flow field 
but at a different time step (or iteration) [7]. At the point where a proces- 
sor has completed the calculation of the first three columns (or rows) of grid 
points, another processor begins calculating the next solution step for the 
grid. Obviously, one of the key advantages of this technique is that the algo- 
rithm does not have to be partitioned for parallel solution. The code is 
merely replicated on each processor. Near-linear speedups are possible as pro- 
cessors are added. Initial tests of the technique have been run on a 2- 
processor CRAY X-MP with a measured speedup of 1.87. This technique and others 
are being investigated with the aim of demonstrating "optimum" combinations of 
algorithms and architectures for important ICFM applications. 

MULTI-STAGE COMPRESSOR FLOW PHYSICS PROGRAM 
A number of the advanced computational technologies that have been dis- 
cussed are being focused on the NASA Lewis Multi-Stage Compressor Flow Physics 
Program. The objective of that program is to increase the fundamental under- 
standing of multi-stage compressor flow physics in order to develop validated, 
practical methods for computing the flows in multistage compressor systems. 

Key elements in this program are the continued development of flow models (the 
average-passage model and a Reynolds-averaged, Navier-Stokes model) and the 
conducting of multi-stage compressor experiments to collect data that can be 
used to further define the models. 
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Figure 14 illustrates the planned use of parallel processing to link and 
enhance both the multi-stage CFD development and the fundamental physical 
experiments. The ERBNET local-area network will provide the connectivity and 
high-speed data communications required to allow researchers to acquire, share, 
and compare analytical and experimental results at their desks and in the test 
cell. If parallel processing can produce 1 to 2 hour turnaround times for run- 
ning multi-stage CFD codes, the experimentalist will be able to use the codes 
during the course of a day's running to explain observations and to predict the 
outcome of the next run. 

Laser anemometry is the measurement method of choice at Lewis for obtain- 
ing high resolution flow data in high-speed rotating machines C8]. Lewis is 
recognized as a world leader in the development and application of computer- 
controlled laser anemometer systems (Fig. 15) for compressor and turbine appli- 
cations. Parallel processing techniques are being developed that will overcome 
the long measurement and data reduction times associated with laser measure- 
ments of the three-dimensional flow fields. This will allow the researcher to 
watch the evolution of flow field displays as the data are collected and to 
detect anomolies in the data, repeating or adjusting the experiment if neces- 
sary, without lengthy delays. 

CONCLUDING REMARKS 

The NASA Lewis Research Center is exploiting advances being made in compu- 
tational technology to develop propulsion system simulations that may eventu- 
ally be capable of accurately predicting the performance, weight, cost, and 
reliability of propulsion components, subassemblies, and systems before hard- 
ware is built. The development of this simulation capability and its practical 
application to propulsion system design will be one of the "grand challenges" 
to be faced in aerospace propulsion during the next decade [93. Through its 
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relationships with the propulsion companies, computer vendors, other research 
laboratories, and academia, NASA is working to ensure that the computing needs 
of the propulsion community are adequately addressed and that the required 
blends of hardware and software technologies are made ready to respond to 
those needs. 

The development of a numerical propulsion system simulator also depends 
on being able to integrate many disciplines (e.g., computational fluid mechan- 
ics, computational structural mechanics, mathematics, computer science, and 
computer engineering) to produce the required analytical models and codes. It 
also requires the development of techniques for effectively calculating, visu- 
alizing, and analyzing the results. This paper has described a number of 
recent initiatives that have led to the formation of multi-disciplinary 
research teams and the establishment of a new computational infrastructure and 
methodology for ICFM research. The combination of these resources will lead 
to an acceleration of progress toward the goal of more efficient, reliable, 
and durable propulsion systems. 
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FIGURE 3. - LEVELS OF COMPLEXITY FOR COMPUTATIONAL ANALYSIS 
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FIGURE 4. - COMPUTER CAPABILITIES AND REQUIREMENTS FOR 
3D HYPERSONIC INLET CALCULATIONS. 
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FIGURE 5. - FUTURE IMPACT OF TECHNOLOGY ON INTERNAL 
COMPUTATIONAL FLUID MECHNANICS. 



FIGURE 6. - SOFTWARE ENGINEERING PRINCIPLES APPLIED TO 
PROTEUS CODE DEVELOPMENT. 
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FIGURE 7. - PROTEUS CODE DEVELOPMENT CYCLE. 
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FIGURE 8. - APPLICATION OF ARTIFICIAL INTELLIGENCE TO COMPU- 
TATIONAL FLUID DYNAMICS. 
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FIGURE 10. - ERBNET LOCAL AREA COMPUTING NETWORK. 



FIGURE 11. - GRAPHICS WORKSTATIONS FOR VISUALIZATION OF 3D 
FLOW FIELD DATA. 
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FIGURE 12 - PARALLEL CALCULATIONS OF SSME FUEL TURBINE 
TEMPERATURE PROFILES USING CRAY X-MP. 
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FIGURE 13. - MULTISTAGE TURBOMACHINERY ANALYSIS USING 
THE HYPERCLUSTER WORKSTATION (4-NODE, 16 PROCESSOR 
CONFIGURATION). 
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FIGURE 14. - PARALLEL COMPUTING FOR VALIDATION OF CFD 
CODES. 



FIGURE 15. - TURBOMACHINERY LASER ANEMOMETRY SYSTEMS. 
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